Skip to content

[SC-7759] Exposing static descriptions in test results#274

Merged
juanmleng merged 7 commits intomainfrom
juan5508/sc-7759/exposing-static-descriptions-in-test-results
Dec 16, 2024
Merged

[SC-7759] Exposing static descriptions in test results#274
juanmleng merged 7 commits intomainfrom
juan5508/sc-7759/exposing-static-descriptions-in-test-results

Conversation

@juanmleng
Copy link
Contributor

@juanmleng juanmleng commented Dec 13, 2024

Internal Notes for Reviewers

Added a doc property to TestResult class to clearly separate the static test documentation from the LLM-generated descriptions. This change helps distinguish between the original test documentation (accessed via result.doc) and the dynamic, descriptions generated by the LLM (accessed via result.description). The doc property contains the test's docstring, providing users with direct access to the test's purpose and behavior.

External Release Notes

Added a doc property to TestResult class to clearly separate the static test documentation from the LLM-generated descriptions. This change helps distinguish between the original test documentation (accessed via result.doc) and the dynamic, descriptions generated by the LLM (accessed via result.description). The doc property contains the test's docstring, providing users with direct access to the test's purpose and behavior.

@juanmleng juanmleng self-assigned this Dec 13, 2024
@juanmleng juanmleng added the internal Not to be externalized in the release notes label Dec 13, 2024
Copy link
Contributor

@johnwalz97 johnwalz97 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm 👏

@github-actions
Copy link
Contributor

PR Summary

This pull request introduces several enhancements and bug fixes to the ValidMind project:

  1. Test Result Documentation:

    • Added a doc attribute to the TestResult class to store documentation related to test results.
    • Updated various functions (build_test_result, _run_composite_test, _run_comparison_test, run_test) to include the doc parameter, ensuring that test documentation is properly captured and passed through the test result pipeline.
  2. ROC Curve Test Enhancements:

    • Modified the test_roc_curve_structure and test_perfect_separation tests in test_ROCCurve.py to handle the updated return type of the ROCCurve function, which now returns a tuple containing RawData and Figure.
    • Added assertions to verify the structure and contents of the RawData object, ensuring it contains expected fields like fpr, tpr, and auc.
  3. Test Execution Improvement:

    • Updated tests/test_unit_tests.py to exit with a failure status if any tests fail, improving the robustness of the test execution process.

These changes aim to enhance the documentation of test results, improve the structure and validation of ROC curve tests, and ensure more reliable test execution.

Test Suggestions

  • Verify that the doc attribute is correctly populated and accessible in TestResult instances.
  • Ensure that the ROCCurve function returns a tuple with RawData and Figure and that the tests correctly validate this structure.
  • Test the exit behavior of the test suite to confirm it exits with a failure status when tests fail.
  • Check that the RawData object contains the expected fields (fpr, tpr, auc) and that these fields are correctly validated in the tests.

@juanmleng juanmleng merged commit a86eda3 into main Dec 16, 2024
5 checks passed
@cachafla cachafla added enhancement New feature or request and removed internal Not to be externalized in the release notes labels Jan 28, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants